A Theoretical Framework for Local Adaptive

Interference in neural networks occurs when learning in one area of the input space causes unlearning in another area. Networks that are less susceptible to interference are referred to as spatially local networks. To obtain a better understanding of these properties, a theoretical framework, consisting of a measure of interference and a measure of network localization, is developed. These measures incorporate not only the network weights and architecture but also the learning algorithm. Using this framework to analyze sigmoidal, multi-layer perceptron (MLP) networks that employ the back-propagation learning algorithm on the quadratic cost function, we address a familiar misconception that single-hidden-layer, sigmoidal networks are inherently non-local by demonstrating that given a su ciently large number of adjustable weights, single-hidden-layer, sigmoidal MLPs exist that are arbitrarily local and retain the ability to approximate any continuous function on a compact domain. From this framework a new multi-objective cost function is developed that includes both the standard quadratic approximation error term and a scalar measure of squared interference. Performing gradient descent on this new cost function provides a localizing algorithm that makes networks less likely to be a ected by interference. This localizing algorithm is applied to Radial Basis Function (RBF) networks and sigmoidal MLP networks to demonstrate that in certain applications where interference is a problem, the localizing algorithm can speed up learning, despite the fact that the cost function upon which the algorithm is built includes two competing objectives.